Papers with sequence classification tasks
Sensitivity as a Complexity Measure for Sequence Classification Tasks (2021.tacl-1)
Copied to clipboard
| Challenge: | Existing complexity metrics provide limited practical insight into complexity differences between tasks. |
| Approach: | They propose a theoretical framework for understanding and predicting the complexity of sequence classification tasks using a new extension of the theory of Boolean function sensitivity. |
| Outcome: | The proposed framework predicts the complexity of sequence classification tasks using a new method . it shows that low-sensitivity functions are easier to learn for LSTMs than lexical classifiers . |
A Symbolic Framework for Evaluating Mathematical Reasoning and Generalisation with Transformers (2024.naacl-long)
Copied to clipboard
| Challenge: | evaluating the generalisability of Transformers to out-of-distribution mathematical reasoning problems is a challenge for many open-source models. |
| Approach: | They propose a method for generating and perturbing detailed derivations of equations at scale, aided by a symbolic engine, and compare their results to sequence classification tasks. |
| Outcome: | The proposed framework outperforms GPT-4, GPT-3.5 and a canon of fine-tuned BERT models in classification tasks . perturbations to input reasoning can reduce their performance by up to 80 F1 points . |
Hyperdecoders: Instance-specific decoders for multi-task NLP (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Recent work in NLP has examined the performance of large pretrained transformer-based models in multi-task settings, where a single model is evaluated on multiple tasks simultaneously. |
| Approach: | They propose a method for multi-tasking using a hypernetwork conditioned on the output of an encoder to generate a unique decoder adaptation for every input instance. |
| Outcome: | The proposed method outperforms previous methods for efficient multi-task fine-tuning and maps from encoder representations to output labels. |
Effect of Post-processing on Contextualized Word Representations (2022.coling-1)
Copied to clipboard
| Challenge: | Post-processing of static embeddings has been shown to improve their performance on both lexical and sequence-level tasks. |
| Approach: | They standardize individual neuron activations using z-score, min-max normalization, and remove top principal components using the all-but-the-top method. |
| Outcome: | The proposed method unwraps vital information present in the representations for both lexical and sequence classification tasks. |
SupCL-Seq: Supervised Contrastive Learning for Downstream Optimized Sequence Representations (2021.findings-emnlp)
Copied to clipboard
| Challenge: | SupCL-Seq extends contrastive learning from computer vision to sequence classification tasks. |
| Approach: | They propose a supervised alternative to Masked Language Modeling (MLM) that extends contrastive learning to sequence optimization in NLP by altering the dropout mask probability in standard Transformer architectures. |
| Outcome: | The proposed method leads to large gains on the GLUE benchmark, including 6% absolute improvement on CoLA, 5.4% on MRPC, 4.7% on RTE and 2.6% on STS-B. |
Why and when should you pool? Analyzing Pooling in Recurrent Architectures (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Various pooling techniques have been shown to improve performance of RNNs on text classification tasks. |
| Approach: | They propose a pooling-based variant that captures interactions among predictive tokens in a sentence. |
| Outcome: | The proposed pooling architecture outperforms non-pooling models on sequence classification tasks. |
SMAB: MAB based word Sensitivity Estimation Framework and its Applications in Adversarial Text Generation (2025.naacl-long)
Copied to clipboard
| Challenge: | a scalable approach to classify text with sensitivity is costly because of exponential time complexity. |
| Approach: | They propose a framework for calculating word-level local and global sensitivities . they use a CHECKLIST-generated sentiment analysis dataset to test their approach . |
| Outcome: | The proposed framework can be used to calculate word-level local and global sensitivities . it improves attacks by 15.58%, while using sensitivity as an additional reward improves . |
Generalized Attention Flow: Feature Attribution for Transformer Models via Maximum Flow (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to feature attributions rely on attention weights and attention weightings. |
| Approach: | They propose a feature attribution method that replaces attention weights with the generalized Information Tensor to enhance the performance of Transformer-based models. |
| Outcome: | The proposed method outperforms state-of-the-art feature attribution methods on sequence classification tasks and provides a more reliable interpretation of Transformer model outputs. |